A storage system is a computer that provides storage service relating to the organization of information on writable persistent storage devices, such as memories, tapes or disks. The storage system is commonly deployed within a network attached storage (NAS) or storage area network (SAN) environment. A SAN is a high-speed network that enables establishment of direct connections between a storage system, such as an application server, and its storage devices. The SAN may thus be viewed as an extension to a storage bus and, as such, an operating system of the storage system enables access to stored information using block-based access protocols over the “extended bus”. In this context, the extended bus is typically embodied as Fibre Channel (FC) or Ethernet media (i.e., network) adapted to operate with block access protocols, such as Small Computer Systems Interface (SCSI) protocol encapsulation over FC or Transmission Control Protocol/Internet Protocol (TCP/IP)/Ethernet.
SCSI is a peripheral input/output (I/O) interface with a standard, device independent protocol that allows different peripheral storage devices, such as disks, to attach to the storage system. In SCSI terminology, clients operating in a SAN environment are initiators that initiate requests and commands for data. The storage system is a target configured to respond to the requests issued by the initiators in accordance with a request/response protocol. The SAN clients typically identify and address the stored information in terms of blocks situated within target disks, or logical units thereof (also referred to within the industry as logical unit numbers or luns).
When used within a NAS environment, the storage system may be embodied as a file server including an operating system that implements a file system to logically organize the information as a hierarchical structure of directories and files on, e.g., the disks. Each “on-disk” file may be implemented as a set of data structures, e.g., disk blocks, configured to store information, such as the actual data for the file. The data blocks are typically organized within a volume block number (vbn) space that is maintained by the file system. The file system may also assign each data block in the file a corresponding “file offset” or file block number (fbn). The file system typically assigns sequences of fbns on a per-file basis, whereas vbns are assigned over a larger volume address space. The file system organizes the data blocks within the vbn space as a “logical volume”; each logical volume may be, although is not necessarily, associated with its own file system. A directory, on the other hand, may be implemented as a specially formatted file in which information about other files and directories are stored.
The file server, or filer, of a NAS system may be further configured to operate according to a client/server model of information delivery to thereby allow many client systems (clients) to access shared resources, such as files, stored on the filer. In the client/server model, the client may comprise an application executing on a computer that “connects” to the filer over a computer network, such as a point-to-point link, shared local area network, wide area network or virtual private network implemented over a public network, such as the Internet. The clients typically communicate with the filer by exchanging discrete frames or packets of data according to pre-defined protocols, such as the TCP/IP. NAS systems generally utilize file-based access protocols; therefore, each client may request the services of the filer by issuing file system protocol messages (in the form of packets) to the file system over the network.
A common type of file system is a “write in-place” file system, an example of which is the conventional Berkeley fast file system. In a write in-place file system, the locations of the data structures, such as index nodes (inodes) and data blocks, on disk are typically fixed. An inode is a data structure used to store information, such as metadata, about a file, whereas the data blocks are structures used to store the actual data for the file. The information contained in an inode may include, e.g., ownership of the file, access permission for the file, size of the file, file type and references to locations on disk of the data blocks for the file. The references to the locations of the file data are provided by pointers, which may further reference indirect blocks that, in turn, reference the data blocks, depending upon the quantity of data in the file. Changes to the inodes and data blocks are made “in-place” in accordance with the write in-place file system. If an update to a file extends the quantity of data for the file, an additional data block is allocated and the appropriate inode is updated to reference that data block.
Another type of file system is a write-anywhere file system that does not overwrite data on disks. If a data block on disk is retrieved (read) from disk into memory and “dirtied” with new data, the data block is stored (written) to a new location on disk to thereby optimize write performance. An example of a write-anywhere file system that is configured to operate on a filer is the Write Anywhere File Layout (WAFL™) file system available from Network Appliance, Inc. of Sunnyvale, Calif. The WAFL file system is implemented within a microkernel as part of the overall protocol stack of the filer and associated disk storage. This microkernel is supplied as part of Network Appliance's Data ONTAP™ storage operating system residing on the filer.
The WAFL file system has the capability to generate a snapshot of its active file system. An “active file system” is a file system to which data can be both written and read, or, more generally, an active store that responds to both read and write I/O operations. Further, it should be noted that “snapshot” is a trademark of Network Appliance, Inc. and is used for purposes of this patent to designate a persistent consistency point (CP) image. A persistent consistency point image (PCPI) is a space conservative, point-in-time read-only image of data accessible by name that provides a consistent image of that data at some previous time. More particularly, a PCPI is a point-in-time representation of a storage element, such as an active file system, file or database, stored on a storage device (e.g., on disk) or other persistent memory and having a name or other identifier that distinguishes it from other PCPIs taken at other points in time. A PCPI can also include other information (metadata) about the storage element at the particular point in time for which the image is taken. The terms “PCPI” and “snapshot” may be used interchangeably through out this patent without derogation of Network Appliance's trademark rights.
A file system, such as the WAFL file system, supports multiple snapshots that are generally created on a regular schedule. Each snapshot is a restorable version of the storage element (e.g., the active file system) created at a predetermined point in time and, as noted, is “read-only” accessible and “space-conservative”. Space conservative denotes that common parts of the storage element in multiple snaphots share the same file system blocks. Only the differences among these various snapshots require extra storage blocks. The multiple snapshots of a storage element are not independent copies, each consuming disk space; therefore, creation of a snapshot on the WAFL file system is instantaneous, since no entity data needs to be copied. Read-only accessibility denotes that a snapshot cannot be modified because it is closely coupled to a single writable image in the active file system. The closely coupled association between a file in the active file system and the same file in a snapshot obviates the use of multiple “same” files. In the example of a WAFL based file system, snapshots are described in TR3002 File System Design for a NFS File Server Appliance by David Hitz et al., published by Network Appliance, Inc. and in U.S. Pat. No. 5,819,292 entitled Method for Maintaining Consistent States of a File System and For Creating User-Accessible Read-Only Copies of a File System, by David Hitz et al., each of which is hereby incorporated by reference as though full set forth herein.
Broadly stated, a snapshot is stored on-disk along with the active file system, and is called into a memory of a filer as requested by an operating system. The on-disk organization of the snapshot and the active file system can be understood from the following description of an exemplary file system inode structure 100 shown in FIG. 1. The inode for an inode file 105 contains information describing the inode file associated with the active file system. In this exemplary file system inode structure, the inode for the inode file 105 contains a pointer that references (points to) an inode file indirect block 110. The inode file indirect block 110 contains a set of pointers that reference inodes 117 which, in turn, contain pointers to indirect blocks 119. The indirect blocks 119 include pointers to file data blocks 120A, 120B and 120C. Each of the file data blocks 120(A-C) is capable of storing, e.g., 4 kilobytes (kB) of data. When the file system generates a snapshot of its active file system, a snapshot inode is generated as shown in FIG. 2. The snapshot inode 205 is, in essence, a duplicate copy of the inode for the inode file 105 of the file system 100 that shares common parts, such as inodes and blocks, with the active file system. For example, the exemplary file system structure 200 includes the inode file indirect blocks 110, inodes 117, indirect blocks 119 and file data blocks 120A-C as in FIG. 1
When a user modifies a file data block, the file system writes the new data block to disk and changes the active file system to point to the newly created block. FIG. 3 shows an exemplary inode file system structure 300 after a file data block has been modified. In this example, file data block 120C is modified to file data block 120C′. As a result, the contents of the modified file data block are written to a new location on disk as a function of the exemplary file system. Because of this new location, the indirect block 319 must be rewritten. Due to this changed indirect block 319, the inode 317 must be rewritten. Similarly, the inode file indirect block 310 and the inode for the inode file 305 must be rewritten.
Thus, after a file data block has been modified the snapshot inode 205 contains a pointer to the original inode file indirect block 110 which, in turn, contains pointers through the inode 117 and indirect block 119 to the original file data blocks 120A, 120B and 120C. The newly written indirect block 319 also includes pointers to unmodified file data blocks 120A and 120B. That is, the unmodified data blocks in the file of the active file system are shared with corresponding data blocks in the snapshot file, with only those blocks that have been modified in the active file system being different than those of the snapshot file.
However, the indirect block 319 further contains a pointer to the modified file data block 120C′ representing the new arrangement of the active file system. A new inode for the inode file 305 is established representing the new structure 300. Note that metadata (not shown) stored in any of the snapshotted blocks (e.g., 205, 110, and 120C) protects these blocks from being recycled or overwritten until they are released from all snapshots. Thus, while the active file system inode for the inode file 305 points to new blocks 310, 317, 319, 120A, 120B and 120C′, the old blocks 205, 110 and 120C are retained until the snapshot is fully released.
Snapshots provide a versatile feature that is essential for data recovery operations, such as backup and recovery of storage elements. However, since snapshots are read-only accessible and their contents cannot be modified, their use may be somewhat limited, particularly for operating systems and applications that do not have a notion of a read-only data store (a read-only file system) and that expect to write metadata at any time that the file system is accessible. This limitation may be overcome by using writable read-only snapshot technique as described in U.S. patent application Ser. No. 10/412,478 entitled Writable Read Only Snapshots, by Vijayan Rajan. A writable, read-only snapshot comprises a read-only “image” (file) residing in a snapshot and a writable virtual disk (vdisk) residing in the active file system. The writable vdisk is a “shadow” image of the snapshot file and, as such, includes an attribute that specifies the snapshot file to be used as a backing store. According to the technique, a write operation directed to the writable read-only snapshot is “trapped” such that the data associated with the operation is stored on the shadow, vdisk image in the active file system. In other words rather than directly accessing the read-only snapshot image of a lun, a client accesses the writable vdisk image, which provides a translucent view of the underlying read-only snapshot image. The writable vdisk is a sparse file containing only that data written by the client (e.g., an initiator in a SAN) to the read-only snapshot image subsequent to a snapshot operation to a volume underlying the lun (vdisk).
To the client, the data retrieved from the writable, read-only snapshot is always the latest data written. The client “sees” the writable vdisk data first (if it exists) and is served that data, the underlying read-only snapshot image being inaccessible for the range of valid data in the writable vdisk. Read-only data from the underlying snapshot image is delivered to the client when no valid data overlying the range exists in the writable vdisk. The underlying snapshot image is accessible and recoverable via a non-translucent path of directly accessing the snapshot image. By this technique, data integrity of a snapshotted lun or vdisk (as an inviolate backup) is preserved.
The writable read only snapshot technique allows many writable vdisks to be “tied” to a single backing store snapshot file. This backing store file may be quite large consuming substantial storage (disk) space. After a period of time, it may be desirable to delete the backing store and free the disk space it consumes. Yet the backing store may not be deleted as long as it is referenced by at least one writable vdisk, even if that vdisk has totally diverged from the backing store. That is, if the backing store is referenced by a writable vdisk, it cannot be deleted even if every data block in the vdisk has been modified from its original state in the backing store.
Previous solutions to this problem have required taking the writable vdisk offline while the backing store is being released. This is undesirable since it creates downtime visible to client applications served by the filer or storage system and substantially decreases system performance. Other proposed solutions have involved restoring the backing store to the active file system using a technique such as single file snap restore (SFSR) described in U.S. patent application Ser. No. 10/100,948 entitled System and Method for Restoring a Single File from a Snapshot. Yet this is not a desirable solution because the restored snapshot does not contain the contents of modified data blocks, such as modified data block 120C′, and rather reflects an older version of the of data blocks, such as old version 120C. Further any technique involving SFSR would generally render the snapshot inaccessible for the duration of the restore operation.
What is needed is a “zero-downtime” technique to delete a snapshot and free its consumed disk space.